<html>
<head>
  <title>An introduction to machine learning with scikit-learn</title>
  <basefont face="微软雅黑" size="2" />
  <meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
  <meta name="exporter-version" content="Evernote Windows/307027 (zh-CN, DDL); Windows/6.1.0 (Win32);"/>
  <style>
    body, td {
      font-family: 微软雅黑;
      font-size: 10pt;
    }
  </style>
</head>
<body>
<a name="951"/>
<h1>An introduction to machine learning with scikit-learn</h1>

<div>
<span><div><h2><span style="font-size: 18pt; color: rgb(28, 51, 135); font-weight: normal;">Model persistence</span></h2><div><ul><li><span style="line-height: 1.45;">It is possible to save a model in the scikit by using Python’s built-in persistence model, namely</span> <a href="https://docs.python.org/2/library/pickle.html" style="line-height: 1.45;">pickle</a><span style="line-height: 1.45;">:</span></li></ul></div><div><br/></div><div><ul><li><span style="line-height: 1.45;">In the specific case of the scikit, it may be more interesting to use joblib’s replacement of pickle (joblib.dump &amp; joblib.load), which is more efficient on big data, but can only pickle to the disk and not to a string:</span></li></ul></div><div><br/></div><h2><span style="font-size: 18pt; color: rgb(28, 51, 135); font-weight: normal;">Conventions</span></h2><div>scikit-learn estimators follow c<font style="font-size: 12pt;"><span style="font-size: 12pt; color: rgb(173, 0, 0); font-weight: bold;">ertain rules to make their behavior more predictive</span></font>.</div><h3>Type casting</h3><div><ul><li><span style="line-height: 1.45;">Unless otherwise specified, input will be cast to float64: fit_transform(X)</span></li></ul></div><div><br/></div><div><ul><li><span style="line-height: 1.45;">Regression targets are cast to float64, classification targets are maintained:</span></li></ul><h3>Refitting and updating parameters</h3></div><div><ul><li><span style="line-height: 1.45;">Hyper-parameters of an estimator can be updated after it has been constructed via the</span> <a href="http://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html#sklearn.pipeline.Pipeline.set_params" style="line-height: 1.45;" title="sklearn.pipeline.Pipeline.set_params">sklearn.pipeline.Pipeline.set_params</a> <span style="line-height: 1.45;">method. Calling fit() more than once will overwrite what was learned by any previous fit():</span></li></ul></div><div style="box-sizing: border-box; padding: 8px; font-size: 12px; border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902);"><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">clf = SVC()clf.set_params(kernel='linear').fit(X, y)</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco; line-height: 1.45;">clf.predict(X_test)</span></div><div><br/></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco; line-height: 1.45;">clf.set_params(kernel='rbf').fit(X, y)</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco; line-height: 1.45;">clf.predict(X_test)</span></div></div><div><br/></div><div>Here, the default kernel rbf is first changed to linear after the estimator has been constructed via SVC(), and changed back to rbf to refit the estimator and to make a second prediction.</div><h3>Multiclass vs. multilabel fitting</h3><div>When using <a href="http://scikit-learn.org/stable/modules/classes.html#module-sklearn.multiclass" title="sklearn.multiclass">multiclass</a> <a href="http://scikit-learn.org/stable/modules/classes.html#module-sklearn.multiclass" title="sklearn.multiclass">classifiers</a>, the learning and prediction task that is performed is dependent on the format of the target data fit upon</div><div><br/></div><div style="box-sizing: border-box; padding: 8px; font-size: 12px; border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902);"><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">&gt;&gt;&gt; from sklearn.svm import SVC</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">&gt;&gt;&gt; from sklearn.multiclass import OneVsRestClassifier</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">&gt;&gt;&gt; from sklearn.preprocessing import LabelBinarizer</span></div><div><br style="font-family: Monaco; font-size: 9pt; color: rgb(51, 51, 51);"/></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">&gt;&gt;&gt; X = [[1, 2], [2, 4], [4, 5], [3, 2], [3, 1]]</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">&gt;&gt;&gt; y = [0, 0, 1, 1, 2]</span></div><div><br style="font-family: Monaco; font-size: 9pt; color: rgb(51, 51, 51);"/></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">&gt;&gt;&gt; classif = OneVsRestClassifier(estimator=SVC(random_state=0))</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">&gt;&gt;&gt; classif.fit(X, y).predict(X)</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">array([0, 0, 1, 1, 2])</span></div></div><div><br/></div><div>In the above case, the classifier is fit on a 1d array of multiclass labels and the predict() method therefore provides corresponding multiclass predictions. It is also possible to fit upon a 2d array of binary label indicators</div><div><br/></div><div style="text-align: center;"><font color="#1C3387" style="font-size: 18pt;"><b>Model selection: choosing estimators and their parameter</b></font></div><h2 style="text-align: center;"><span style="font-size: 18pt; color: rgb(28, 51, 135);">Score, and cross-validated scores</span></h2><div>        As we have seen, every estimator exposes a score method that can judge the quality of the fit (or the prediction) on new data. <span style="font-weight: bold;">Bigger is better</span>.</div><div>        To get a better measure of prediction accuracy (which we can use as a proxy for goodness of fit of the model), we can successively split the data in <span style="font-style: italic;">folds</span> that we use for training and testing:</div><div style="box-sizing: border-box; padding: 8px; font-size: 12px; border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902);"><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">import numpy as np</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">X_folds = np.array_split(X_digits, 3)</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">y_folds = np.array_split(y_digits, 3)</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">scores = list()</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">for k in range(3):</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">    # We use 'list' to copy, in order to 'pop' later on</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">    X_train = list(X_folds)</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">    X_test = X_train.pop(k)</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">    X_train = np.concatenate(X_train)</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">    y_train = list(y_folds)</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">    y_test = y_train.pop(k)</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">    y_train = np.concatenate(y_train)</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">    scores.append(svc.fit(X_train, y_train).score(X_test, y_test))</span></div><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">print(scores)</span></div></div><div><br/></div><h2><span style="font-size: 18pt; color: rgb(28, 51, 135);">Cross-validation generators</span></h2><div><span style="font-size: 12pt; color: rgb(173, 0, 0); font-weight: bold;">Scikit-learn has a collection of classes which can be used to generate lists of train/test indices for popular cross-validation strategies</span></div><div><br/></div><ul><li><span style="line-height: 1.45;">The</span> <span style="font-size: 12pt; color: rgb(173, 0, 0); font-weight: bold; line-height: 1.45;">cross-validation score</span> <span style="line-height: 1.45;">can be directly calculated using the</span> <a href="http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_val_score.html#sklearn.model_selection.cross_val_score" style="line-height: 1.45;" title="sklearn.model_selection.cross_val_score">cross_val_score</a> <span style="line-height: 1.45;">helper.</span></li></ul><div><br/></div><div>        Given an estimator, the cross-validation object and the input dataset, the <a href="http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_val_score.html#sklearn.model_selection.cross_val_score" title="sklearn.model_selection.cross_val_score">cross_val_score</a> splits the data repeatedly into a training and a testing set, trains the estimator using the training set and computes the scores based on the testing set for each iteration of cross-validation.</div><ul><li>n_jobs=-1 means that the computation will be dispatched on all the CPUs of the computer.</li><li>Alternatively, the scoring argument can be provided to specify an alternative scoring method.</li></ul><div><br/></div><div><br/></div><h2><span style="font-size: 18pt; color: rgb(28, 51, 135);">Grid-search and cross-validated estimators</span></h2><h3>Grid-search</h3><div>        scikit-learn provides an object that, given data, computes the score during the fit of an estimator on a parameter grid and chooses the parameters to maximize the cross-validation score. This object takes an estimator during the construction and exposes an estimator API:</div><div>      <span style="font-size: 12pt; color: rgb(173, 0, 0); font-weight: bold;">  调试每一种参数，以求得到最好的score</span></div><div style="box-sizing: border-box; padding: 8px; font-size: 12px; border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902);"><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">clf = GridSearchCV(estimator=svc, param_grid=dict(C=Cs),n_jobs=-1)</span></div></div><div><br/></div><h3>Cross-validated estimators</h3><div>        Cross-validation to set a parameter can be done more efficiently on an algorithm-by-algorithm basis. This is why, for certain estimators, scikit-learn exposes <a href="http://scikit-learn.org/stable/modules/cross_validation.html#cross-validation">Cross-validation: evaluating estimator performance</a> estimators that set their parameter automatically by cross-validation</div><div style="box-sizing: border-box; padding: 8px; font-size: 12px; border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902);"><div><span style="font-size: 9pt; background-color: rgb(251, 250, 248); color: rgb(51, 51, 51); font-family: Monaco;">lasso = linear_model.LassoCV()</span></div></div><div><br/></div><div>These estimators are called similarly to their counterparts, with ‘CV’ appended to their name.</div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div><div><br/></div></div><div><br/></div></span>
</div></body></html> 